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POSITIVE RECURRENCE OF REFLECTING BROWNIAN 
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University of Minnesota, Georgia Institute of Tecnology and Stanford 

University 

Consider a semimartingale reflecting Brownian motion (SRBM) 
Z whose state space is the d-dimensional nonnegative orthant. The data 
for such a process are a drift vector 9, a nonsingular dx d covariance 
matrix E, and a d x d reflection matrix R that specifies the boundary 
behavior of Z. We say that Z is positive recurrent, or stable, if the 
expected time to hit an arbitrary open neighborhood of the origin is 
finite for every starting state. 

In dimension d = 2, necessary and sufficient conditions for stability 
are known, but fundamentally new phenomena arise in higher dimen- 
sions. Building on prior work by El Kharroubi, Ben Tahar and Yaa- 
coubi [Stochastics Stochastics Rep. 68 (2000) 229-253, Math. Methods 
Oper. Res. 56 (2002) 243-258], we provide necessary and sufficient 
conditions for stability of SRBMs in three dimensions; to verify or 
refute these conditions is a simple computational task. As a byprod- 
uct, we find that the fluid-based criterion of Dupuis and Williams 
[Ann. Probab. 22 (1994) 680-702] is not only sufficient but also nec- 
essary for stability of SRBMs in three dimensions. That is, an SRBM 
in three dimensions is positive recurrent if and only if every path of 
the associated fluid model is attracted to the origin. The problem 
of recurrence classification for SRBMs in four and higher dimensions 
remains open. 

1. Introduction. This paper is concerned with the class of d-dimensional 
diffusion processes called semimartingale reflecting Brownian motions (SRBMs), 
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which arise as approximations for open ci-station queueing networks of var- 
ious kinds; cf. Harrison and Nguyen (1993) and Williams (1995, 1996). The 
state space for a process Z = {Z(t),t > 0} in this class is S = Mi (the non- 
negative orthant). The data of the process are a drift vector 9, a nonsingular 
covariance matrix V, and a d x d "reflection matrix" R that specifies bound- 
ary behavior. In the interior of the orthant, Z behaves as an ordinary Brow- 
nian motion with parameters 9 and T, and roughly speaking, Z is pushed 
in direction R? whenever the boundary surface {z G S : Zj = 0} is hit, where 
R J is the jih column of R, for j = 1, . . . ,d. To make this description more 
precise, one represents Z in the form 

(1.1) Z(t) = X(t) + RY(t), t>0, 

where X is an unconstrained Brownian motion with drift vector 9, covariance 
matrix T, and Z(0) = X(0) € S, and Y is a ti-dimensional process with 
components Y\ , . . . , such that 

(1.2) Y is continuous and nondecreasing with Y(0) = 0, 

(1.3) Yj only increases at times t for which Zj(t) = 0, j = 1, . . . ,d, and 

(1.4) Z(t)eS, t>0. 

The complete definition and essential properties of the diffusion process Z 
will be reviewed in Appendix A, where we also discuss the notion of posi- 
tive recurrence. As usual in Markov process theory, the complete definition 
involves a family of probability measures {P^x G S} that specify the distri- 
bution of Z for different starting states; informally, one can think of P x ( - ) 
as a conditional probability given that Z(0) = x. Denoting by E x the expec- 
tation operator associated with F x and setting ta = inf{i > 0:Z(t) £ A}, 
we say that Z is positive recurrent if E x (ta) < oo for any x G S and any 
open neighborhood A of the origin (see Appendix A for elaboration). For 
ease of expression, we use the terms "stable" and "stability" as synonyms 
for "positive recurrent" and "positive recurrence," respectively. 

In the foundational theory for SRBMs, the following classes of matrices are 
of interest. First, a d x d matrix R is said to be an S-matrix if there exists 
a d-vector w > such that Rw > (or equivalently, if there exists w > 
such that Rw > 0), and R is said to be completely-S if each of its principal 
submatrices is an 5-matrix. (For a vector v, we write v > to mean that each 
component of v is positive, and we write v > to mean that each component 
of v is nonnegative.) Second, a square matrix is said to be a P-matrix if all of 
its principal minors are positive (that is, each principal submatrix of R has a 
positive determinant). "P-matrices are a subclass of completely-5 matrices; 
the still more restrictive class of .M-matrices is defined as in Chapter 6 of 
Berman and Plemmons (1979). References for the following key results can 
be found in the survey paper by Williams (1995): there exists a diffusion 
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process Z of the form described above if and only if R is a completely S 
matrix; and moreover, Z is unique in distribution whenever it exists. 

Hereafter we assume that R is completely-5. Its diagonal elements must 
then be strictly positive, so we can (and do) assume without loss of generality 
that 

(1.5) Ra = 1 for alH = 1, . . . , d. 

This convention is standard in the SRBM literature; in Sections 5 through 
7 of this paper (where our main results are proved) another convenient nor- 
malization of problem data will be used. Appendix B explains the scaling 
procedures that justify both (1.5) and the normalized problem format as- 
sumed in Sections 5 through 7. 

We are concerned in this paper with conditions that assure the stability 
of Z. An important condition in that regard is the following: 

(1.6) R is nonsingular and R~ l 6 < 0. 

If R is an -matrix, then (1.6) is known to be necessary and sufficient for 
stability of Z; Harrison and Williams (1987) prove that result and explain 
how the .M-matrix structure arises naturally in queueing network applica- 
tions. 

El Kharroubi, Tahar and Yaacoubi (2000) further prove the following three 
results: first, (1.6) is necessary for stability in general; second, when d = 2, 
one has stability if and only if (1.6) holds and R is a "P-matrix; and third, 
(1.6) is not sufficient for stability in three and higher dimensions, even 
if R is a P-matrix. In Appendix C of this paper, we provide an alter- 
native proof for the first of these results, one that is much simpler than 
the original proof by El Kharroubi, Tahar and Yaacoubi (2000). Appendix 
A of Harrison and Hasenbein (2009) contains an alternative proof of the 
second result. Section 3 of this paper reviews the ingenious example by 
Bernard and El Kharroubi (1991) that serves to establish the third result, 
that an SRBM can be unstable, cycling to infinity even if (1.6) holds; Theo- 
rem 4 in this paper, together with the examples provided in Section 6, shows 
that instability can also occur in other ways when (1.6) holds. 

A later paper by El Kharroubi, Ben Tahar and Yaacoubi (2002) estab- 
lished sufficient conditions for stability of SRBMs in three dimensions, rely- 
ing heavily on the foundational theory developed by Dupuis and Williams 
(1994). In this paper, we show that the conditions identified by 
El Kharroubi, Ben Tahar and Yaacoubi (2002) are also necessary for sta- 
bility when d = 3; the relevant conditions are easy to verify or refute via 
simple computations. As a complement to this work, an alternative proof 
of the sufficiency result by El Kharroubi, Ben Tahar and Yaacoubi (2002) is 
also being prepared for submission; cf. Dai and Harrison (2009). 
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The remainder of the paper is structured as follows. First, to allow precise 
statements of the main results, we introduce in Section 2 the "fluid paths" 
associated with an SRBM, and the linear complementarity problem that 
arises in conjunction with linear fluid paths. That section, like the paper's 
first three appendices, considers a general dimension d, whereas all other 
sections in the body of the paper consider d = 3 specifically. Section 3 iden- 
tifies conditions under which fluid paths spiral on the boundary of the state 
space S. Section 4 states our main conclusions, which are achieved by com- 
bining the positive results of El Kharroubi, Ben Tahar and Yaacoubi (2002) 
with negative results that are new; Figure 2 in Section 4 summarizes suc- 
cinctly the necessary and sufficient conditions for stability when d = 3, and 
indicates which components of the overall argument are old and which are 
new. In Sections 5 through 7, we prove the new "negative results" referred to 
above, dealing first with the case where fluid paths spiral on the boundary, 
and then with the case where they do not. As stated above, Appendix A 
reviews the precise definition of SRBM; Appendix B explains the scaling 
procedures that give rise to normalized problem formats and Appendix C 
contains a relatively simple proof that (1.6) is necessary for stability. Fi- 
nally, Appendix D contains several technical lemmas that are used in the 
probabilistic arguments of Section 7. 

2. Fluid paths and the linear complementarity problem. 

Definition 1. A fluid path associated with the data (0,R) is a pair of 
continuous functions y, z: [0, oo) — > W 1 that satisfy the following conditions: 



Definition 2. We say that a fluid path (y,z) is attracted to the origin 
if z{t) — > as t — > oo. 

Definition 3. A fluid path (y,z) is said to be divergent if \z(t)\ — > oo 
as t — > oo, where, for a vector u = (u{) £ M. d , \u\ = ^ i 

Theorem 1 [Dupuis and Williams (1994)]. Let Z be a d- dimensional 
SRBM with data (6,T,R). If every fluid path associated with (0,R) is at- 
tracted to the origin, then Z is positive recurrent. 



(2.1) 
(2.2) 
(2.3) 



z(t) = z(0) +9t + Ry(t) for all t > 0, 
z(t) G S for all t > 0, 

y(-) is continuous and nondecreasing with y(0) = 0, 
yj(-) only increases when Zj(-) = 0, i.e., 



(2.4) 




d). 
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Definition 4. A fluid path (y,z) is said to be linear if it has the form 
y(t) = ut and z{t) = vt, t> 0, where u, v > 0. 

Linear fluid paths are in one-to-one correspondence with solutions of the 
following linear complementarity problem (LCP): Find vectors u = (m) and 
v = (vi) in R d such that 

(2.5) u,v>0, 

(2.6) v = 9 + Ru, 

(2.7) u-v = 0, 

where u ■ v = UiVi is the inner product of u and v. [See Cottle, Pang and Stone 
(1992) for a systematic account of the theory associated with the general 
problem (2.5)-(2.7).] 

Definition 5. A solution (u, v) of the LCP is said to be stable if v = 
and to be divergent otherwise. It is said to be nondegenerate if u and v 
together have exactly d positive components, and to be degenerate otherwise. 
A stable, nondegenerate solution of the LCP is called proper. 

Lemma 1. Suppose that (1.6) holds. Then (u*,0) is a proper solution of 
the LCP, where 

(2.8) u* = -R~ 1 6, 

and any other solution of the LCP must be divergent. 

Proof. The first statement is obvious. On the other hand, for any stable 
solution (it, 0) of the LCP, we have from (2.6) that 9 + Ru = 0; since (1.6) 
includes the requirement that R be nonsingular, u = —R~ 1 9 = u* . That is, 
there cannot exist a stable solution other than (u*,0), which is equivalent 
to the second statement of the lemma. □ 



3. Fluid paths that spiral on the boundary. Bernard and El Kharroubi 
(1991) devised the following ingenious example with d = 3, referred to here- 
after as the B&EK example: let 





and R = 

This reflection matrix R is completely S (moreover, it is a P-matrix), so 
Z is a well-defined SRBM. (The covariance matrix V is immaterial to the 
discussion that follows, provided only that it is nonsingular.) As 
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Bernard and El Kharroubi (1991) observed, the unique fluid path with these 
process data, starting from z(0) = (0, 0, re) with re > 0, is the one pictured in 
Figure 1; it travels in a counter-clockwise and piecewise linear fashion on the 
boundary, with the first linear segment ending at (2re,0,0), the second one 
ending at (0,4re, 0), and so forth. El Kharroubi, Tahar and Yaacoubi (2000) 
proved that an SRBM with these data is not stable, showing that if re is 
large then \Z(t) — z(t)\ remains forever small (in a certain sense) with high 
probability. 

To generalize the B&EK example, let C\ be the set of (0, R) pairs that 
satisfy the following system of inequalities [here Rij denotes the (i,j)th 
element of R, or equivalently, the ith element of the column vector R 3 ]: 



[Notation used in this section agrees with that of El Kharroubi, Tahar and 
Yaacoubi (2000, 2002) in all essential respects, but is different in a few minor 
respects.] 

To explain the meaning of these inequalities, we consider a fluid path 
associated with (6,R) that starts from z(0) = (0,0, re), where re > 0; it is the 
unique fluid path starting from that state, but that fact will not be used 
in our formal results. Over an initial time interval [0,ti], the fluid path is 
linear and adheres to the boundary {z 2 = 0}, as in Figure 1. During that 
interval one has y(t) = (0, — 02, 0) and hence the fluid path has the constant 



(3.1) 
(3.2) 
(3.3) 
(3.4) 



3 > 0ii? 3 i and 2 < 0ifl2i. 



01 > 2 Ru and 3 < 6 2 R 32 



2 > 3 i?23 and 0i < 9 3 R 13 



0<O 



4k 



2k 




Fig. 1. Fluid model behavior of the B&EK example. 
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velocity vector 

(6\ — 62R12 ' 

63 — 62R32 , 

Thus (3.1) and (3.2) together give the following: as in Figure 1, a fluid 
path starting from state (0,0, k) has an initial linear segment in which z% 
decreases, z\ increases and Z2 remains at zero; that initial linear segment 
terminates at the point z(t\) on the z\ axis that has 

( \ f Qi~ #2-^12 \ ^ n 

Similarly, from (3.1) and (3.3), the fluid path is linear over an ensuing 
time interval [t\ , T2] , with z\ decreasing, Z2 increasing and 2:3 remaining at 
zero; that second linear segment terminates at the point z{j2) on the Z2 axis 
that has 

£2(72) ~~ ( ^ 1 ~ ® 2 ^ 12 \ ( ® 2 — ^ 3 ^ 23 j K > 

V #2-^32 — #3 / V #3-^13 — OlJ 

Finally, from (3.1) and (3.4), the fluid path is linear over a next time interval 
b^jTs]) with Z2 decreasing, Z3 increasing and z\ remaining at zero; that 
third linear segment terminates at the point z{r%) on the Z3 axis that has 
^3(73) =Pi(0,R)k, where 

(3.6) m R) = ( ( 6 a 2 ~ d3R f ) ( 9 a 3 ~ 9lR f ) > 0. 



#2-^32 — #3 / V #3-^13 — 8l J \ O1R2I — #2 

Thereafter, the piecewise linear fluid path continues its counter-clockwise 
spiral on the boundary in a self-similar fashion, like the path pictured in 
Figure 1, except that in the general case defined by (3.1) through (3.4), the 
spiral may be either inward or outward, depending on whether f3i(6,R) < 1 
or Pi(0,R) > 1. 

To repeat, C\ consists of all (9,R) pairs that satisfy (3.1) through (3.4), 
and the single-cycle gain f3\(9,R) for such a pair is defined by (3.6). As we 
have seen, fluid paths associated with problem data in C\ spiral counter- 
clockwise on the boundary of S. Now let C2 consist of all (0,R) pairs that 
satisfy (3.1) and further satisfy (3.2) through (3.4) with all six of the strict 
inequalities reversed. It is more or less obvious that (6,R) pairs in C2 are 
those giving rise to clockwise spirals on the boundary, and the appropriate 
analog of (3.6) is 

fo(e,R) 



(3.7) 



#3 — #2^32 \ / 8l — #3-^13 \ / 82 — O1R2I \ 
O2R12 — OiJ \O3R23 — 62 J \0lRz\ — O3 J 



>0. 
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Hereafter we define C = d U C 2 , P(9,R) = Pi(d,R) for (0,i?) G d and 
0(9, R) = I3 2 (0,R) for (0,R) G C 2 . Thus C consists of all (6,R) pairs whose 
associated fluid paths spiral on the boundary, and 0(9, R) is the single-cycle 
gain for such a pair. 

4. Summary of results in three dimensions. Theorem 2 below is a slightly 
weakened version of Theorem 1 by El Kharroubi, Ben Tahar and Yaacoubi 
(2002), which the original authors express in a more elaborate notation; 
we have deleted one part of their result that is irrelevant for current pur- 
poses. The corollary that follows is immediate from Theorem 1 above (the 
Dupuis-Williams fluid stability criterion) and Theorem 2. 

Theorem 2 [El Kharroubi, Ben Tahar and Yaacoubi (2002)]. Suppose 
that (1.6) holds and that either of the following additional hypotheses is 
satisfied: (a) (9,R) G C and (3(6, R) < 1; or (b) (0,R) £ C and the linear 
complementarity problem (2.5)-(2.7) has a unique solution, which is the 
proper solution (u*,0) defined in (2.8). Then all fluid paths associated (6,R) 
are attracted to the origin. 

Corollary. Suppose that (1.6) holds and, in addition, either (a) or 
(b) holds. Then Z is positive recurrent. 

The proof of Theorem 2 in El Kharroubi, Ben Tahar and Yaacoubi (2002) 
is not entirely rigorous, containing verbal passages that mask significant 
technical difficulties; an alternative proof that uses a linear Lyapunov func- 
tion to prove stability is given in Dai and Harrison (2009). 

The new results of this paper are Theorems 3 and 4 below, which will be 
proved in Sections 5 through 7. Figure 2 summarizes the logic by which these 
new results combine with previously known results to provide necessary and 
sufficient conditions for stability (i.e., positive recurrence) of Z. 

Theorem 3. If (9,R) G C and (3(6, R) > 1, then Z is not positive recur- 
rent. 

Theorem 4. Suppose that (1.6) is satisfied. If there exists a divergent 
solution for the linear complementarity problem (2.5)-(2.7), then Z is not 
positive recurrent. 

5. Proof of Theorem 3. Throughout this section and the next, we assume 
without loss of generality that our problem data satisfy not only (1.5) but 
also 



(5.1) 



0;G {-1,0,1} fori = 1,2, 3. 
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R A e <o ? 



Yes 



Z is not stable (El Kharroubi et al. 2000; 

see Appendix C for an alternative proof) 

(u * 0) is a proper solution of the LCP, and if any other 
solution exists, it must be divergent (Lemma 1) 



6£C(R) ? 



Yes 



I3(B,R) < 1 



Yes Z is stable (Theorem 2a, 
> El Kharroubi et al. 2002) 



No 



No 



-> Z is not stable (Theorem 3) 



Is there a divergent 
solution of the LCP? 



Yes 



No 



Z is not stable (Theorem 4) 



Z is stable (Theorem 2b, 



El Kharroubi et al. 2002) 

Fig. 2. Summary of results in three dimensions. 

Appendix B explains the scaling procedures that yield this normalized form. 
To prove Theorem 3 we will assume that (#, R) € C\ and /3\(9, R) > 1, then 
show that Z is not stable; the proof of instability when (9, R) 6 C2 and 
02(0, R) > 1 is identical. Given the normalizations (1.5) and (5.1), the con- 
ditions (3.1) through (3.4) that define C\ can be restated as follows: 



(5.2) 
(5.3) 



9 = (-1,-1,-1)', 

^12,^23,-^31 > 1 an d i?i3 , i?21 , i?32 < 1 • 



Let us now define a 3 x 3 matrix V by setting Vij = Rij — 1 for i,j = 1, 2, 3. 
Then V 3 (the jth column of V) is the vector 9 — OjR 3 for j = 1,2,3. Note 
that V 2 was identified in (3.5) as the velocity vector on the face {Z2 = 0} 
for a fluid path corresponding to (9,R). 

Lemma 2. Under the assumption that f3i(0,R) > 1, there exists a vector 
u > such that u'V > 0, or equivalently, u'V 3 > for each j = 1, 2, 3. 



Proof. From (1.5) and (5.3) we have that 





[ ° 


a 2 


-63 


(5.4) 


V=\-h 





03 




V 01 


-62 
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where di,bi > for i = 1,2,3. In this notation, the definition (3.6) is as 
follows: 

(5-5) ^ R ) = TTT- 

01 »2 03 

Setting 

aia 2 , a 2 
ui = l, u 2 = — — and u 3 = —, 

0\b 2 2 

it is easy to verify that u'V 1 = u'V 2 = 0, and u'V 3 = 6 3 (^f§ - 1). The def- 
inition (5.5) and our assumption that (3i(9,R) > 1 then give u'V 3 > 0. □ 

For the remainder of the proof of Theorem 3, let e denote the three- vector 
of ones, so (5.2) is equivalently expressed as 8 = — e, and we can represent 
X in (1.1) as 

(5.6) X(t) =X(0) + B(t) -et, t>0, 

where B is a driftless Brownian motion with nonsingular covariance matrix 
and B(0) = 0. Also, we choose a starting state x = X(0) = Z(0) that satisfies 

(5.7) Zi(0)>0, Z 2 {0)=0 and Z 3 {0)>0. 

In this section, because the initial state is fixed, we write E(-) rather than 
Ea;(-) to signify the expectation operator associated with the probability 
measure (see Appendix A). Also, when we speak of stopping times and 
martingales, the relevant filtration is the one specified in Appendix A. 

Let u > be chosen to satisfy u'V > 0, as in Lemma 2, and further 
normalized so that u'e = 1. It is immediate from the definition of V that 
u'V = u'R — e' , and thus one has the following: 

(5.8) u'R>e'. 

Now define £{t) = u'Z(t), t > 0. From (1.1), (5.6) and (5.8), one has 
f (i) - f (0) = u'Bti) - u'et + u'RY(t) 

(5.9) 

>u'B(t)-t + e'Y(t) fort>0. 

Next, let 

n = inf{t > : Z 3 (t) = 0}, r 2 = inf{t > n : Z x (t) = 0}, 

T 3 = inf{t>T2:Z 2 (t) = 0} 

and so forth. (These stopping times are analogous to the points in time at 
which the piecewise linear fluid path in Figure 1 changes direction.) The cru- 
cial observation is the following: Z 3 (-) > over the interval [0, ri), Z\{-) > 
over [ti,t 2 ), Z 2 {-) > over [t 2 ,t 3 ) and so forth. Thus Y 3 (-) does not increase 
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over [0, Ti), Yi(-) does not increase over [ri,r2), Y 2 {-) does not increase over 
[ T 2>7"3) and so forth. 

From (1.1) and (5.6), we then have the following relationships: 

Z 2 (t) = B 2 {t)-t + Y 2 (t) 

(5.10) 

+ RnY 1 (t), 0<t< n , 
Z 3 (t) = [B 3 (t) - B 3 (n)] - (t - n) + [Y 3 (t) - y 3 (ri)] 

(5.11) 

+ ^ 32 [>2(t)->2(ri)], ri<t<r 2) 
Zi(t) = [Si(t) - Si(r 2 )] - (t - r 2 ) + [Yi(t) - Fi(n)] 

(5.12) 

+ Rl 3 [Y 3 (t)-Y 3 (T 2 )}, T 2 <t<T 3 . 

There exist analogous representations for Z 2 over the time interval [t3,T4], 
for Z 3 over [t4,T5], for Z\ over [t5,T6], and so on. Now (5.10) gives 

(5.13) Y 2 (t) = t + Z 2 (t)-B 2 (t)-R 21 Y 1 (t) for0<t<n. 

Because Y 3 = on [0, t\), we can substitute (5.13) into (5.9) to obtain the 
following: 

t(t)-m>u , B(t)-t+Y 1 (t) 

(5.14) 

+ [t + Z 2 (t) - B 2 (t) - R^Yl (t)] for < t < n- 

From the definition of V and (5.4), we have 1 — R 2 \ = b\ > 0, so (5.14) can 
be rewritten 

(5.15) - f(0) > M(t) + A{t) for0<t<n, 
where 

(5.16) M{t) = u'B{t)-B 2 {t) for0<t<n, 

(5.17) A(t) = Z 2 (t) +6iYi(t) for0<t<n. 

Defining r = limr n , we now extend the definition (5.16) to all i 6 [0,t) as 
follows: 

M(t) = M(n) + u'[B(i) - B(n)] 

- [B 3 (i) - BsCn)] fom<i<T 2 , 
M(t) = M(r 2 ) + u'[B{t) - B(t 2 )] 

-[B 1 (t)-B 1 (r 2 )] for r 2 < t < r 3 

and so forth. Finally, on {r < oo}, we set M(t) = M(r) for all t>r. Then 
M = {M(t),t > 0} is a continuous martingale whose quadratic variation 
(M,M)(-) satisfies 

(5.20) (M,M)(t) - (M,M)(s) <7(t-a) for < s < t < oo, 



(5.18) 
(5.19) 
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where < 7 < 00. Also, we extend (5.17) to all t G [0, r) via 

(5.21) A(t) = A(T 1 ) + Z 3 {t) + b 2 [Y 2 (t)-Y 2 ( Tl )] for n<t<T 2 , 

(5.22) A(t)=A(T 2 ) + Z 1 (t) + b 3 [Y 3 (t)-Y 3 (T 2 )} for r 2 < t < r 3 

and so forth. Thus the process A = {A(t),0 < t < r} is nonnegative and 
continuous. 

Lemma 3. f (i) - £(0) > M(i) + /or a// 1 G [0, r) . 

Proof. It has already been shown in (5.15) that this inequality is valid 
for < t < t\. In exactly the same way, but using (5.11) instead of (5.10), 
one obtains 

(5.23) at)-an) = [M(t)-M(r 1 )] + [A(t)-A(T 1 )] for n<t<r 2 , 

so the desired inequality holds for < t < r 2 . Continuing in this way, the 
desired inequality is established for < t < r. □ 

To complete the proof of Theorem 3, let T = inf{i > : = e} and let 
a = inf{t > 0:£(0) + M(t) = e}, where < e < ^(0). From Lemma 3, the 
nonnegativity of A(-), and the fact that £(r) = on {r < 00}, we have the 
following inequalities: < a < T < r. Thus it suffices to prove that E(cr) = 
00, which can be shown by essentially the same argument that applies when 
M is an ordinary (driftless) Brownian motion. That is, we first let a(b) = 
m£{t > : f (0) + M(t) = e or f (0) + M (t) = b}, where b > ^(0) . Because both 
M and M 2 - (M, M) are martingales and (5.20) holds, one has < E[a(b)] < 
00, E[M(a(b))\ =0 and E[M 2 (a(b))\ = E[(M, M)(a(b))]. It follows by the 
optional sampling theorem that 

E[(M,M)(a(b)))=E[M 2 {a(b))] 

(5-24) =(b- m f^ + m -ef b -j^ 

= (&-£(0))(£(0)-e). 

The left-hand side of (5.24) is < jE[a(b)] by (5.20), the right-hand side f 00 
as b t 00, and obviously a > o(b) for all b > £(0). Thus E(cr) = 00, and the 
proof of Theorem 3 is complete. 

6. Categories of divergent LCP solutions. Our goal in the remainder of 
the paper is to prove Theorem 4. We continue to assume the canonical prob- 
lem format in which R satisfies (1.5) and 6 satisfies (5.1). In the following 
lemma and later, the term "LCP solution" is used to mean a solution (u,v) 
of the linear complementarity problem (2.5)-(2.7). 
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Lemma 4. If (1.6) holds, then (a) 9 > is not possible, and (b) there 
exists no LCP solution (u,v) with v > 0. 



Proof. Because R is completely S by assumption, its transpose is also 
completely S; cf. Proposition 1.1 of Dai and Williams (1995). Thus there 
exists a vector a > such that a'R > 0. Now (1.6) says that 9 + Ry = for 
y > 0. Multiplying both sides of the equation by a' and rearranging terms, 
one has a'9 = —a'Ry < 0, which implies conclusion (a). Also, if (u,v) is a 
LCP solution with v > 0, one has from (2.7) and (2.6) that u = and v = 9, 
which contradicts conclusion (a). This implies conclusion (b). □ 



We now define five nonoverlapping categories of divergent LCP solutions. 
Immediately after each category is defined, we shall exhibit a pair (R,9) 
which admits a LCP solution (u,v) in that category, or else direct the reader 
to a proposition that shows the category to be empty. Readers may verify 
that the reflection matrix R appearing in each of our examples is completely 
S. Also, defining u* = —R~ 1 9 as in (2.8), we shall display the vector u* for 
each example, showing that u* > and hence (1.6) is satisfied. 



Category I. Exactly two components of v are positive, and the com- 
plementary component of u is positive. The following is such an example: 



R 



u 









)■ -1 









Category II. Exactly one component of v is positive, det(i?) > 0, and 
the two complementary components of u are not both zero, where R is the 
2x2 principal submatrix of R corresponding to the two zero components of 
v. Such an example is given by 





1/2 






1 






1 















). ,.| 




|. -1 
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Here, the two complementary components of u are both positive. In the 
following example, which also falls in Category II, just one of them is posi- 
tive: 



R 



u 



Category III. Exactly one component of v is positive, det(i?) = 0, and 
the two complementary components of u are not both zero. In Lemma 8, it 
will be shown that no such LCP solutions exist if (1.6) holds. 

Category IV. Exactly one component of v is positive, det(-R) < 0, and 
the two complementary components of « are both positive. Such an example 
is given by 



R 



( 19/68 \ 
u* = 15/34 , 
V 2/17 / 

It will be shown in Lemma 7 that if there exists a LCP solution in Cat- 
egory IV, under our restrictions on R and 9, there also exists a solution 
in Category I or Category II (or both). For the example above, a second 
LCP solution is (u,v), where u = (0,1,0)' and v = (1/10,0,1)'; this second 
solution lies in Category I. 

Category V. Exactly one component of v is positive, det(-R) < 0, and 
exactly one of the two complementary components of u is positive. Such an 
example is given by 



R 




u 



1 


1 


-2/5 


2 


1 


-6/5 


-2 - 


■1/10 




9/8 ' 






5/14 


) 


■-( 


45/28 











). ..| 


V 
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Lemma 5. Suppose that (1.6) holds and that (u,v) is a divergent LCP 
solution. Then (u, v) belongs to one of the five categories defined immediately 
above. 

Proof. Let m and n denote the number of positive components in u and 
v, respectively; the complementarity condition (2.7) implies that m + n< 3. 
Lemma 4 shows that n < 3 (i.e., v > cannot hold); also n > 0, because 
(u, v) is a divergent LCP solution by assumption. Thus, either n = 1 or 
n = 2. Moreover, it is not possible that m = 0, or equivalently u = 0, because 
then (2.5) and (2.6) would imply 9 = v > 0, which contradicts Lemma 4. 
So the only remaining possibilities are (m,n) = (1,2), (m,n) = (2,1) and 
(m,n) = (1,1). Category I is precisely the case where (m,n) = (1,2), and 
Categories II through V together cover the cases where (m,n) = (2,1) and 



It will be shown in Section 7 that Z cannot be positive recurrent if there 
exists a LCP solution in Category I, Category II or Category V. Lemma 7 
in this section will show that the existence of a LCP solution in Category 
IV implies the existence of a LCP solution in either Category I or Category 
II. Lemma 8 at the end of this section will show that LCP solutions in 
Category III cannot occur when (1.6) holds. In combination with Lemma 5 
above, these results obviously imply Theorem 4. 

We now state and prove Lemma 6, which we need in order to prove 
Lemma 7. Our scaling convention (1.5) specifies that R has ones on the 
diagonal, so we can write 



for some constants a, a', b,b', c and d . 

Lemma 6. Assume that there does not exist a LCP solution in Category 
I, and that there is a divergent LCP solution (u,v) with u\ > 0, U2 > 0, 
us = 0, vi = v 2 = and v% > 0. Let R be as in (6.1) and assume that the 
principal submatrix R corresponding to the zero components of v satisfies 
det( J R)<0. Then 9 = (-1,-1,1)' anda,a'>l. 

Proof. Because (u,v) is a solution of the LCP (2.5)-(2.7), one has 



(m,n) = (1,1). □ 



(6.1) 




(6.2) 
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Because v\ = v% = and U3 > 0, 

1 a' 



(6.3) R 

1 a 1 

Setting u = (1*1,1*2)' and = (61,62)', we have from (6.2) that 

(6.4) Ru = -6. 
Because R is an 5-matrix with negative determinant, 

(6.5) a, a' > and act > 1. 

Because ui > and 1*2 > by hypothesis, it is immediate from (6.4) and (6.5) 
that both components of 6 are negative, so our canonical rescaling gives 6 = 
(-1,-1)'. Thus, either 5 = (-1,-1,-1)', (-1,-1,0)' or (-1,-1,1)' must 
hold. From (6.5), (6.4) and = (-l,-l)', it follows that 

(6.6) a,a'>l. 

We will show that 6 = (—1, —1, 1)' by excluding the other two cases. Sup- 
pose first that 6 = (—1, —1, —1)'. Then (6.2) becomes 



(6.7) 



It must be true that b, b' < 1; otherwise there would be a solution of the 
LCP that falls into Category I. For example, if b > 1 one then has a di- 
vergent LCP solution (u,v) with u = (1,0,0)', v\ = 0, V2 = o — 1 > and 
V3 = b— 1 > 0. However, one cannot have a, a' >l,b,b'<l and (6.7) holding 
simultaneously, which gives a contradiction. 

Next suppose that 6 = (—1,-1,0)'. Here, (6.7) holds with V3 in place of 
1 + V3 on the right-hand side. We must have b, b 1 < 0, for the same reason 
as before. This results in a contradiction, and the only remaining possibility 
under our canonical rescaling is 6 = (—1, —1, 1)'. □ 





Lemma 7. If there exists a LCP solution in Category IV, then there also 
exists a solution in Category I or Category II (or both). 

Proof. Denoting by (u,v) a solution in Category IV, we assume that 
no solution in Category I exists. It will then suffice to prove that a solution 
in Category II exists. By permuting the indices, we can assume that u\ > 0, 
U2 > 0, U3 = 0, v\ = V2 = and v-j > 0. 

We use the notation in (6.1). By Lemma 6, one has a, a' > 1 and 6 = 
(—1,-1,1)'. We shall assume that c' > c, then construct a LCP solution 
(u,v) that falls into Category II, with u\ > , M3 > 0, V2 > 0; in exactly the 
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same way, if c > d, one can construct a LCP solution (u,v) that falls into 
Category II, with v± > , u 2 > , U3 > 0. 

We first observe that b,b' < — 1. Otherwise, contrary to the assumption 
imposed in the first paragraph of the proof, there would exist a LCP solution 
in Category I. For example, if b > —1, then there is a divergent LCP solution 
(u, v) with u = (1, 0, 0)', v\ = 0, v 2 = a - 1 > 0, and -u 3 = b + 1 > 0. 

The 2x2 submatrix of R that is relevant to our construction is 



R 



1 c 
b 1 



Because i? is an 5-matrix and b < 0, we know that be < 1, hence det(i?) > 0, 
and because 6 < — 1 we also know that c > — 1. Letting 7 = (71,72)' be the 
two- vector satisfying Rj = (1,-1)', one has 

1 ( 1 -c\ ( 1 \ 1 ( 1 + c 

7 



Defining u = (71 , 0, 72)' and {; = 9 + -Rn, it follows that v\ = V3 = 0. Compar- 
ing the first and second rows of R term by term, and noting that the first 
two components of 9 are identical, one sees that 

(6.8) V2-vi = r^7-[( a " !)(! + c ) " ( c ' " c )(! + &)]■ 

1 — oc 

Because of the inequalities a > 1, c > — 1, c' > c and b < — 1, the quantity 
inside the square brackets in (6.8) is positive. Thus Vi > 0, and hence (u,v) 
is a LCP solution in Category II. □ 



Lemma 8. If (1.6) holds, then there cannot exist a LCP solution in 
Category III. 



Proof. Arguing as in the proof of Lemma 7, we assume the existence 
of a LCP solution (u, v) in Category III. By permuting the indices, we can 
assume that u\ > 0, u 2 > 0, U3 = 0, v\ = v 2 = and V3 > 0. We use the 
notation (6.1) and define R by (6.3). A minor variation of the first paragraph 
in the proof of Lemma 6 shows for the current case that both 9\ and 9 2 are 
negative, and so 9\ = 9 2 = — 1 with our scaling convention. One then has 
a = a' = 1 in (6.1), because det(i?) = by assumption. By (1.6), 9 + Ru* = 
for some u* > 0, from which it follows that c = d in (6.1). That is, the first 
two rows of R are identical, whereas (1.6) includes the requirement that R 
be nonsingular. □ 



18 



M. BRAMSON, J. G. DAI AND J. M. HARRISON 



7. Proof of Theorem 4. As we explained immediately after the proof of 
Lemma 5 in Section 6, the proofs of Lemmas 9, 10 and 12 in this section 
will complete the proof of Theorem 4. In Lemmas 9 and 10, we actually 
prove that Z is transient, which is stronger than we require for Theorem 4. 
The SRBM Z is said to be transient if there exists an open ball C centered at 
the origin such that P{tc = oo} > for some initial state Z{fS) = i£ IR^_ that 
is outside of the ball, where tq = inf{i > 0:Z(t) G C}. Clearly, when Z is 
transient, it is not positive recurrent. In this section, we continue to assume 
the canonical problem format in which R satisfies (1.5) and satisfies (5.1). 

Lemma 9. If there is a LCP solution (u,v) in Category I, then Z is 
transient. 

Proof. Without loss of generality, we assume that 

(7.1) ui>0, u 2 = 0, it3 = 0, vi = 0, v 2 >0, v 3 >0. 

Because v = + Ru, with R as in (6.1), 9\ < 0, and so, by our scaling 
convention, 6\ = — 1. It follows from this that 

(7.2) ui = l, v 2 = 2 + a>0 and v 3 = 6 3 + b>0. 
One can write (1.1) as 

(7.3) Z 1 (t) = Z 1 {0) + e 1 t + B 1 (t) + Y 1 (t) + a'Y 2 (t) + cY 3 (t) , 

(7.4) Z 2 (t) = Z 2 (0) + 6 2 t + B 2 (t ) + aYi (t ) + Y 2 (t ) + c% (t) , 

(7.5) Z 3 (t) = Z 3 (0) + e 3 t + B 3 (t) + bY 1 (t) + b'Y 2 (t) + Y 3 (t) 

for t > 0, where = {£?(£), i > 0} is the three-dimensional driftless Brow- 
nian motion with covariance matrix T. Assume Z(Q) = (0,N,N)' for some 
constant N > 1 and set r = inf{t > : Z 2 (t) = 1 or Z 3 (t) = 1}. We will show 
that P{r = oo} > for sufficiently large N, and thus Z is transient. Because 
01 = -1 and Y 2 (t) = Y 3 (t) = for t £ [0,r), one has, for i < r, 

Z 1 (t) = -t + B 1 (t) + yi(i), 

Z 2 (t) = N + 2 t + B 2 (t) + aFi (t) , 

Z 3 (t) = 7V + 03t + J B 3 (t) + 6y 1 (t). 

For i > 0, let Y\(t) = sup 0<s<t (— s + B\(s))~ , and set 

(7.6) Z 1 (t) = -t + Si(t) + ^i(t), 

(7.7) Z 2 (i) = A r + 2 t + 5 2 (t) + ayi(t), 

(7.8) Z 3 {t) = N + 6 3 t + B 3 {t) + bY l (t) 



REFLECTING BROWNIAN MOTION 19 

for t > 0. Clearly, Z(t) = Z{t) for t S [0, r]. In particular, r = f, where f = 
inf{t > : Z 2 {t) = 1 or Z 3 {t) = 1}. To show Z is transient, it suffices to prove 
that, for sufficiently large N, P{f = oo} > 0. 

By the functional strong law of large numbers (FSLLN) for a driftless 
Brownian motion, one has 

lim — sup |B(ts)| =0 almost surely. 

t 0< S <1 

This implies that 

lim t _1 Yi(t) = lim t" 1 sup (-s + Bi(s))~ 

i^oo <->oo Q<s<t 

= lim sup (-s + t" 1 ^!^))^ 

'^°°0<s<l 

= sup (— s)~ = 1 almost surely. 

0<s<l 

Therefore, by (7.2), (7.7) and (7.8), one has ]imt-Hx>t ^(t) = v 2 > and 

limt_>oo t~ l Zs{t) = V3 > almost surely. Consequently, there exists a con- 
stant T > such that F(A) > 3/4 for all N>1, where 

(7.9) A = {Z 2 (t) > 1 and Z 3 (t) > 1 for all i > T}. 
One can choose iV large enough so that ¥(B) > 3/4, where 

(7.10) B = {Z 2 (t) > 1 and Z 3 (t) > 1 for all t G [0,T]}. 
Because AnBcjf = 00}, P{f = oo}>l/2>0, as desired. □ 

Lemma 10. If there is an LCP solution (u,v) in Category II, then Z is 
transient. 

Proof. Without loss of generality, we assume that 

(7.11) ui>0, u 2 >0, u 3 = 0, vx = 0, v 2 = 0, v 3 >0. 

Assume R is given by (6.1), and let R be the 2x2 principal submatrix of R 
given by (6.3). By assumption, det(i?) > 0. One can check that conditions 
(2.5)-(2.7) and (7.11) imply that 

(7.12) WM<0, 

(7.13) v 3 = 6 3 -(b,b')R- 1 ( d A >0. 
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Let Z(0) = (0, 0, N)' for some constant N > 1 and set r = m£{t > : Z 3 (i) = 
1}. We will show that for sufficiently large N, P{r = oo} > 0, and thus Z is 
transient. 

On {t < t}, one has Z 3 (t) > and thus Y 3 (t) = 0. Because the SRBM Z 
satisfies Equation (1.1), on {t < r}, 

Z(t) = Z(0) + 9t + B{t) + R Y 2 {t) 

(7.14) 

Yi(t)\ 

= Z{0) + 9t + B{t) + R\ Y 2 (t) , 



V^(t)/ 




where 

R 



One can check that because R is completely S, so is i?. It therefore follows 
from Taylor and Williams (1993) that there exists a SRBM Z associated 
with the data (Rj_,#,r,ii) that starts from Z(0). Following Definition 6 
in Appendix A, the three-dimensional process Z, together with the corre- 
sponding processes B and Y, is defined on some filtered probability space 
(fi,{J^},P); B, Y and Z are adapted to {Ft}, and P-almost surely satisfy 
(1.1)-(1.4); B is a driftless Brownian motion with covariance matrix T, and 
B is an {J^j-martingale. Furthermore, from Taylor and Williams (1993), 
the distribution of Z is unique. Because of (7.14), B, Y and Z also satisfy 
(1.1)-(1.4) on {t < t}, with the same data 0, T, R), and so r = f in 
distribution, where 

f = inf{i>0:Z 3 (t) = l}. 
We now show that for sufficiently large N, 

(7.15) P{f = oo}>0, 

which implies that P{r = oo} > 0. We note that (Z\, Z 2 ) is a two-dimensional 
SRBM with data 0, f , R), where 6 = {0 U 2 )' and f is the 2 x 2 principal 
submatrix of V obtained by deleting the 3rd row and the 3rd column of T. 
Lemma 14 in Appendix D will show that when R is a P-matrix and the 
condition (7.12) is satisfied, the two-dimensional SRBM Z is "rate stable" 
in the sense that 

(7.16) lim -ZAt) =0 almost surely, i = 1,2. 

t—too t 
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Solving for Y\ and Y 2 in the first two components of (1.1) and plugging 
them into the third component yields 

Z 3 (t) = N + 9 3 t + B 3 (t) 

(7.17) 

+ (b,b')R- 

Equations (7.13), (7.16) and (7.17), together the SLLN for Brownian motion, 
imply that 

liminf — Z 3 (t) > v 3 almost surely. 

t— too t 

Because v 3 > 0, one can argue as in (7.9) and (7.10) that for N large enough, 
F{Z 3 (t) > 1 for all t > 0} > 0. This proves (7.15). □ 

Before stating and proving Lemma 12 for a LCP solution in Category V, 
we state the following lemma, which is needed in the proof of Lemma 12 
and will be proved at the end of this section. 

Lemma 11. Let B = (B±, B2, B 3 ) be a three-dimensional Brownian mo- 
tion with zero drift and covariance matrix T, starting from 0. Set 

(7.18) Z 1 (t) = -t + B 1 (t) + Y 1 (t), 

(7.19) Z 2 (t) =2 + B 2 (t)- Si(t) + Zi(t), 

(7.20) Z 3 (t) =4N + 3/xt + B 3 (t) + aB^t) - aZ x {t) 
for t>0, and given constants a, fi > and N > 1, where 

Y 1 (t)= sup (-s + B 1 (s)y. 

0<s<t 

Then for sufficiently large N , one has E(a) = oo ; where a = inf{t > : Z 2 {t) = 
1 orZ 3 (t) = l}. 

Lemma 12. If there is a LCP solution (u,v) in Category V, then Z is 
not positive recurrent. 

Proof. Without loss of generality, we assume that 

(7.21) m>0, u 2 = 0, n 3 = 0, vi=0, v 2 = 0, v 3 >0. 

Then a minor variation of the first paragraph in the proof of Lemma 6 
establishes that both 0\ and 9 2 are negative, so 6\ = 9 2 = — 1 with our scaling 
convention. Assuming R is as in (6.1), it follows as in (6.2) that u\ = 1, a = 1 
and v 3 = b + 6 3 > 0. 



Z 1 (t) 



Bi(t) 
B2{t) 



Ys(t), 



t > 0. 
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Let Z(0) = (0, 2, N)' for some constant N > 1 and let 

Ti = mf{t>0:Zi(t) = l}, i = 2,3, 

with r = min(r2,T3). We will show that E x (r) = oo for sufficiently large N, 
which implies that Z is not positive recurrent. 

The SRBM Z satisfies equations (7.3)-(7.5). Since Z 2 (t) > and Z 3 (t) > 
for t < r, one has Y 2 (t) = Y 3 (t) = for t < r. Because a = 1, (7.3)-(7.5) 
reduce to 

(7.22) Z 1 (t) = -t + 5 1 (t) + Yi(t), 

(7.23) Z 2 (t) = 2-t + S 2 (t) + yi(t), 

(7.24) Z 3 (t) = N + 9 3 t + B 3 (t) + bY 1 (t) 

on t<r. By (7.22), one has Y^t) = Z^t) + t - B^t) for t < r. Substituting 
Yi(i) into (7.23) and (7.24), one has 

Z 2 (t) = 2 + B 2 (t)-B 1 (t) + Z 1 (t), 

Z 3 (t) =N + v 3 t + B 3 (t) - 65i(t) + 6Zi (t) 

on t < t. 

For each t > 0, let Yi(t) = sup 0<s<t (— s + Bi(s))~ , and set 

Zi(t) = -t + Si(t) + fi(t), 

Z 2 (t) = 2 + B 2 (t)-5i(t) + Zi(t), 

Z 3 (t) = N + v 3 t + B 3 (t) - bB^t) + 6Zi(t) 

for t > 0. Let f = inf{t > : Z 2 (t) = 1 or ^(t) = 1}; clearly, r = f on every 
sample path. It follows from Lemma 11 that E(t) = oo for sufficiently large 
N. Therefore, E x (r) = oo, and so Z is not positive recurrent. □ 

Proof of Lemma 11. We first prove the case when a > 0. When a < 0, 
the proof is actually significantly simpler; an outline for this case will be 
presented at the end of this proof. 

Let X 2 (t) = 1 + B 2 (t) - Bi{t) and X 3 (t) = B 3 (t) + aB 2 (t) for t > 0. Then 
X 2 is a Brownian motion starting from 1, X 3 is a Brownian motion starting 
from 0, and (7.19)-(7.20) become 

(7.25) Z 2 (t) = l + X 2 (t) + Z 1 (t) >1 + X 2 (t), 
Z 3 (t) = (N + a) + (N + [it + X 3 (t)) 

(7.26) 

+ (2N + 2fit - aX 2 (t) - aZi(t)) 
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for t > 0. Define 

n = inf{i>0:X 2 (t) <0}, 

r 2 = inf{i > : aX 2 (t) > 2N + 2jrf - aZi(i) or aX 2 (t) >N + fit}, 
T 3 = ini{t>0:X 3 (t)<-(N + fit)}, 

T = T\ A T 2 A T3. 

Assuming iV + a > 1, it follows from the definition of r and (7.25)— (7.26) 
that for each t < r, Z 2 {t) > 1 and Z$(t) > 1, and so r < a. To prove the 
lemma, it therefore suffices to show that 

(7.27) E(t) = oo. 

Since X2 is a driftless Brownian motion, E(ri) = 00. When N is large, it 
is intuitively clear that t\ > r 2 A T3 with only negligible probability, which 
leads to E(t) = 00. To make the argument rigorous, first note that, because 
Z\ is adapted to B±, each n is a stopping time with respect to the filtration 
generated by the Brownian motion B, and hence r is a stopping time as 
well. Because X 2 (0) = 1 and X 2 is a martingale with respect to the filtration 
generated by B, by the optional sampling theorem, 

E(X 2 (r At)) = l 

for each t > 0. We will show that, for sufficiently large N, 

(7.28) E(X 2 (r)l {T<oo} )<i. 
It follows that 

l = E(X 2 (rAt)) 
= E(X 2 (T)l {T<t} )+E(X 2 (t)l {T > t} ) 
< E(X 2 (r)l {T<oo} ) + E(((N + fit)/a)l {T > t} ) 

= ± + ((N + nt)/a)F{r>t}, 

where we have used X 2 {t) < (N + fj£) /a on t < r 2 for the inequality. Conse- 
quently, 

(7.29) P{r >t}> — — 

for each t > 0, from which E(r) = 00 follows. 

It remains to prove (7.28). Because X 2 {ji) = when t\ is finite, 

E(X 2 (r)l {T<oo} ) = E(X 2 (r 2 A T3)l {T2AT3<ri} ) 

(7.30) < E(((N + ^(t, A r 3 ))/a)l {T2AT3<oo} ) 



<Y2((N + /i(n + l))/a)P{n < r 2 A r 3 < n + 1}. 



n=0 
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To bound the probability ¥{n < t 2 A r 3 < n + 1} for each n £ Z+, we use 

{n < t 2 A r 3 < n + 1} C {n < t 2 < n + 1} U {n < r 3 < n + 1}. 

For {n < T3 < n + 1}, X 3 (i) < —(N + /in) first occurs on (n,n + 1]. By the 
strong Markov property for Brownian motion and the reflection principle, 
the probability of the latter event is at most 2¥{X 3 (n + 1) < — (N + fin)}. 
For {n < r 2 < n + 1}, either aX2(t) > N + un first occurs on (n,n + 1], or 
&Zi (t) > N + fin occurs on [n, n + 1] . One can also apply the strong Markov 
property and the reflection principle to the first event. One therefore obtains 

P{n < r 2 A r 3 < n + 1} < F{n < t 2 < n + 1} + P{n < r 3 < n + 1} 

<2P{X 3 (n+l) <-(N + un)} 

(7.31) 

+ 2P{aX 2 (n + 1) > N + an} 



+ p{ sup aZi{s) > N + un\. 



We proceed to bound each of these three terms and plug these bounds 
into the last term in (7.30). Note that 

P{X 3 (n + 1) < —(TV + fin)} = p( jV(0, 1) > E±EL 

I 7v n + 1 



(7.32) 1 ^y_nTl exp /_l(iV + ^ 



V^F N + fin ' \ 2 7 2 (n + l) 
< — =^exp( -^{N + fin) 

for N > fx, where iV(0, 1) denotes the standard normal random variable 
and 7 2 is the variance of the Brownian motion X 3 . The first inequality is a 
standard estimate and is obtained by integrating by parts. Assume N = N'fi, 
with iV'eZ + . Then, 



(7.33) 



J2( N + Ufa + l))2F{X 3 (n + 1) < -(N + fin)} 

n=0 

s ?,vfe ( " +1,exp ("^ ( " +1) )' 



which is less than 1/6 for sufficiently large N. For the same reason, 

00 1 
(7.34) V(JV + u(n + l))2P{aX 2 (n + 1) > TV + fin} < - 

n 6 

n=0 
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for sufficiently large N. To bound the probability IP{sup n<s < n+1 aZ\ (s) > 
N + /xn}, we apply Lemma 13 in Appendix D. The lemma states that for 
appropriate constants ci,c 2 > 0, 

F< sup > x > < c\ exp(— c 2 x) for x > 0. 

For AT = iV'u, 



oo 




oo 

< (n + l)/ici exp(-c 2 /in/a), 

n=AT' 

which is also less than 1/6 for large AT. The bounds obtained for (7.33)— (7.35) 
together show that the last term in (7.30) is at most 1/2. This implies (7.28), 
and hence the lemma for a > 0. 

When a < 0, the proof is analogous to the case a > 0, with the following 
simplifications. Assume N + a > 1. The equality (7.26) can be replaced by 

Z 3 (t) >{N + a) + (N + vt + X 3 (t)) + (2N + - aX 2 {t)) 

for £ > 0, because a < 0. The definition of r 2 can be replaced by the simpler 

r 2 = inf{t > 0:X 2 (i) > A r + /xt}. 

One can again check that for each t < r, Z 2 (t) > 1 and ^(t) > 1, and so 
t < a. To prove the lemma for the case a < 0, it remains to show (7.27). 
For this, we follow the same procedure as in the case a > 0. First, (7.28) 
still implies (7.29) with a in the right side of (7.29) replaced by 1. From 
(7.29), (7.27) follows. To demonstrate (7.28), we employ (7.30), with a there 
replaced by 1. To bound the probability P{ri < r 2 A t 3 < n + 1} for each 
n G Z+, we replace (7.31) with the simpler 

P{n < r 2 A r 3 < n + 1} 

< P{n < r 2 < n + 1} + P{ra < r 3 < n + 1} 

< 2P{X 3 (n + 1) < —(N + fin)} + 2P{A" 2 (n + 1)>N + fin}. 

It then follows from bounds (7.32)-(7.34) that (7.28) holds for sufficiently 
large N. This implies the lemma for a < 0. □ 
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APPENDIX A: SEMIMARTINGALE REFLECTING BROWNIAN 



In this section, we present the standard definition of a semimartingale 
reflecting Brownian motion (SRBM) in the d-dimensional orthant S = M+, 
where d is a positive integer. We also review the standard definition of pos- 
itive recurrence for an SRBM, connecting it with the alternative definition 
used in Section 1. 

Recall from Section 1 that 9 is a constant vector in M. d , V is a d x d 
symmetric and strictly positive definite matrix, and R is a d X d matrix. 
We shall define an SRBM associated with the data (S,9,T,R). For this, a 
triple (Q, J 7 , {Ft}) will be called a filtered space if O is a set, J 7 is a cr-field of 
subsets of and {Ft} = {Ft,t > 0} is an increasing family of sub-cr-fields 
of F, that is, a filtration. 

Definition 6 (Semimartingale reflecting Brownian motion). A SRBM 
associated with (S,9,T,R) is a continuous {J^j-adapted d-dimensional pro- 
cess Z = {Z(t), t > 0}, together with a family of probability measures {f x ,x S 
S}, defined on some filtered space (Q,F, {Ft}) such that, for each x G S, 
under F x , (1.1) and (1.4) hold, where, writing W(t) = X(t) - 9t for t > 
0, W is a d-dimensional Brownian motion with covariance matrix T, an 
{J^j-martingale such that W(0) =x P^-a.s., and Y is an {J^j-adapted d- 
dimensional process such that P^-a.s. (1.2) and (1.3) hold. Here (1.2) is 
interpreted to hold for each component of Y, and (1.3) is defined to be 



Definition 6 gives the so-called weak formulation of a SRBM. It is a stan- 
dard definition adopted in the literature; see, for example, Dupuis and Williams 
(1994) and Williams (1995). Note that condition (A.l) is equivalent to the 
condition that, for each t > 0, Zj (t) > implies Yj (t — 6) = Yj it + 5) for some 
5 > 0. Reiman and Williams (1988) showed that a necessary condition for 
a (S, 9, r, i?)-SRBM to exist is that the reflection matrix R is completely 
S (this term was defined in Section 1). Taylor and Williams (1993) showed 
that when R is completely S, a (S, 9, T, i?)-SRBM Z exists and Z is unique 
in law under ¥ x for each x £ S. Furthermore, Z, together with the family 
of probability measures {P x ,x G is a Feller continuous strong Markov 
process. 

Let (9,T,R) be fixed with T being a positive definite matrix and R being 
a completely S matrix. Dupuis and Williams [(1994), Definition 2.5] and 
Williams [(1995), Definition 3.1] gave the following definition of positive 
recurrence. 



MOTIONS 



(A.l) 




for all t > 0. 
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Definition 7. An SRBM Z is said to be positive recurrent if, for each 
closed set A in S having positive Lebesgue measure, we have K x (ta) < oo 
for all x G S, where r A = M{t > : Z(t) G A}. 

Because each open neighborhood of the origin contains a closed ball that 
has positive volume, Definition 7 appears to be stronger (i.e., more restric- 
tive) than the definition adopted in Section 1, but one can show that these 
two notions of positive recurrence are equivalent for a SRBM. Indeed, the 
last paragraph on page 698 in Dupuis and Williams (1994) provides a sketch 
of that proof. 

APPENDIX B: CONVENIENT NORMALIZATIONS OF PROBLEM 

DATA 

Let R be a d x d completely S matrix and (X, Y, Z) a triple of continuous, 
d-dimensional stochastic processes defined on a common probability space. 
The diagonal elements of R are necessarily positive. Let D = diag(-R) and 
R = RD~ l (thus R is a d x d completely-5 matrix that has ones on the 
diagonal), and define Y(t) = DY{t) for t > 0. If (A, Y, Z) satisfy (1.1)-(1.4) 
with reflection matrix R, then (X , Y, Z) satisfy (1.1)— (1.4) with reflection 
matrix R, and vice versa. Thus the distribution of Z is not changed if one 
substitutes R for R, and that substitution assures the standardized problem 
format (1.5). 

Now let R and (X, Y, Z) be as in the previous paragraph, and further 
suppose that X is a Brownian motion with drift vector 8 and nonsingular 
covariance matrix V. Define a d x d diagonal matrix D by setting Da = 1 
if 0i = and Da = \8i\~ 1 otherwise. Setting Z = DZ, X = DX and R = 
DR, one sees that if (X, Y, Z) satisfy (1.1)— (1.4) with reflection matrix 
R, then (X , Y, Z) satisfy (1.1)-(1.4) with reflection matrix R, and vice 
versa. Of course, A" is a Brownian motion whose drift vector 9 = DO satisfies 
(5.1); the covariance matrix of X is T = DTD. Thus our linear change of 
variable gives a transformed problem in which (5.1) is satisfied. To achieve a 
problem format where both (1.5) and (5.1) are satisfied, one can first make 
the linear change of variable described in this paragraph, and then make the 
substitution described in the previous paragraph. 

APPENDIX C: PROOF THAT (1.6) IS NECESSARY FOR STABILITY 

OF Z 

We consider a d-dimensional SRBM Z with associated data (S,6,T,R), 
defined as in Appendix A, assuming throughout that R is completely S. Let 
us also assume until further notice that R is nonsingular. Because R is an 
5-matrix, there exist cf-vectors w, v > such that Rw = v. That is, 

(C.l) irt>0 where v>0. 
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Now suppose it is not true that R 9 < 0. That is, defining 7 = R 6, 
suppose that 7, > for some i G {1, . . . ,d}. For future reference let u be the 
ith. row of R~ l . Thus (C.l) implies 

(C.2) u-v>0. 

Our goal is to show that Z cannot be positive recurrent. Toward this end, 
it will be helpful to represent the underlying Brownian motion X in (1.1) 
as X(t) = W(t) + 9t, where W is a d-dimensional Brownian motion with 
zero drift and covariance matrix T. Premultiplying both sides of (1.1) by 
R" 1 then gives R~ l Z{t) = R~ l W(t) + 7* + Y(t). The ith component of the 
vector equation is 

(C.3) i{t) = u-Z{t) = u-W{t)+-iit + Yi{t), t>0. 

Let A = {z G S: \z\ < 1} and B = {u ■ z:z G A}. Then B C R is a compact 
interval containing the origin, and from (C.2) we know that B contains 
positive values as well, because A contains av for sufficiently small constants 
a > 0. Thus B has the form 

(C.4) B = [a, b] where a < and b > 0. 

As the initial state x = Z(0) = W(0), we take 

x = f3v where v is chosen as in (C.l) and 

(C.5) 

> max(|u| ,(u-v) b). 

From (C.l), (C.2), (C.4) and (C.5), we have that 

(C.6) x G S, \x\ > 1 and u- x > b. 

Thus, defining r A = M{t > 0:Z(t) G A} and a = inf{t > 0:£(t) G B}, it 
follows from the definitions of A, B and ^, plus (C.4) and (C.6), that 

(C.7) T A > a, Pa.-a.8- 

From (C.3), we see that ^ is bounded below by a one-dimensional Brown- 
ian motion with nonnegative drift, and £(0) > b P x -a.s. Thus K x (a) = 00, 
implying that ~E x (ta) = 00 as well by (C.7). This establishes that Z is not 
positive recurrent when R is nonsingular. 

We still need to show that Z cannot be positive recurrent when R is 
singular. In this case, there exists a nontrivial vector u G M. d such that u'R 
= 0, and we can assume that u ■ 9 > as well (because — u can be exchanged 
for u if necessary). Premultiplying both sides of (1.1) by v! gives the following 
analog of (C.3): 

u- Z{t)=u-W{t) + (u-9)t. 
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Because R is an 5-matrix, for this given u there exist w,v £ S such that 

(C.8) u + Rw = v. 

After premultiplying both sides of (C.8) by v! , one obtains 

i 1 9 

u ■ v = \u\ > 0. 

We choose the initial state x = Z(0) = W(0) exactly as in (C.5), and define 
the set A as before. The proof that E x (ta) = oo, and hence that Z is not 
positive recurrent, then proceeds exactly as in the case treated above, except 
that now the process £ = u ■ Z is itself a Brownian motion with nonnegative 
drift, whereas in the case treated earlier £ was bounded below by such a 
Brownian motion. 



APPENDIX D: TWO LEMMAS 

In this appendix, we demonstrate two lemmas that are used in Section 7. 
Lemma 13 is employed in the proof of Lemma 11. 



Lemma 13. Let X be a one- dimensional Brownian motion with drift 6 < 
0, variance a 2 , starting from 0. Let Z be the corresponding one- dimensional 
SRBM, 

Z(t) = X(t) - min X(s) for t > 0. 

0<s<t 

There exist constants c\ > and C2 > such that 
(D.l) Pj sup Z(s) >x\ <ciexp(-c 2 x) 

^n— l<s<n ' 

for all n E Z + and x > 0. 



Proof. One could employ the Lipschitz continuity property of the one- 
dimensional Skorohod map and the estimate (4.9) of Atar, Budhiraja and Dupuis 
(2001) to prove the lemma. Here, we provide a direct proof. Since X has 
negative drift, it is well known [see, e.g., Section 1.9 of Harrison (1985)] that, 
for each t > and x > 0, 

P{Z(t) > x} = pf max (X(t) - X(t - a)) > x\ 
(D.2) =p{ max X(s) > x\ 

<P{ sup X(s) >x\ =exp(-2|#|x/cr 2 ). 
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Therefore, for n G Z+, 

max Z(s) > x > 

^n—l<s<n J 

<P{Z(n)>x/2}+p\z(n)<x/2, max Z(s)>x\ 

I n—l<s<n ) 



J2\ 



< exp(-\6\x/<r) + P{n - 1< r < n,X(n) - X(t) < -x/2}, 

where r = inf{s > n — 1 : Z{s) > x}. 

Note that r is a stopping time with respect to the filtration generated 
by Z, which is the filtration generated by the Brownian motion B, where 
B(t) = X(t) - 0t for t > 0. By the strong Markov property of B, 

¥{n - 1< r < n, X(n) - X(t) < -x/2} 

< ¥{n - 1< r < n,B(n) - B(r) < -x/2 + |0|} 

<$((-x/2 + \0\)/a) 

for x > 2\6\, where $ is the standard normal distribution function. Thus, 

V< max Z(s) >x\< exp(-|6»|x/cj 2 ) + $((-x/2 + I) /a) 
Ln— Ks<n J 

for x > 2\0\, from which (D.l) follows. □ 

The following lemma is used in the proof of Lemma 10. Let 



R = [ 1 ? ) and 9 



a 1 ) \ 6 



and assume that det(i?) > 0. Then R is a "P-matrix and hence is completely 

5. For any given 2x2 positive definite matrix T, it therefore follows from 
Taylor and Williams (1993) that, starting from any fixed state x G M?., there 
is a two-dimensional SRBM Z with data (0,T,R), as defined in Definition 

6, that is well defined and is unique in law. 

Lemma 14. Suppose that R _1 < 0. Then each fluid path (y,z) starting 
from z(0) = remains at 0; that is, z{t) = for t>0. Consequently, 

Z(t) 

(D.3) lim = almost surely. 

t—¥00 t 

Proof. It follows from 

1 — ab \— a 1 
det(-R) = 1 - ab > and R~ l < that 

01 - b0 2 < 0, 2 - a0i < 0. 
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Observe that R is a P-matrix because R is a P-matrix. Therefore, R 
is an 5-matrix. Consequently, there exists (^1,^2) > such that (01,02) = 
(di,d2)R~ 1 > 0. By assumption R _1 8 < 0, and so one has 

c 1 e 1 + c 2 2 = (d 1 ,d 2 )R- 1 e<0. 

Let (y,z) be a fluid path with z(0) = 0. By the oscillation inequality [see, 
e.g., Lemma 4.3 of Dai and Williams (1995)], (y,z) is Lipschitz continuous. 
Setting g[t) = c\Z\(t) + c 2 z 2 (t), we will show that g(t) = for t > 0. From 
this, it follows that z(t) = for t > 0, which is the first claim in the lemma. 
It suffices to prove that 

g(t) < at each t where g(t) > and (y, z) is differentiable. 

We consider several cases, depending on whether z±(t) and z 2 (t) are strictly 
positive. When z x (t) > and z 2 (t) > 0, (2.1)~(2.4) imply that z x (t) = 61 
and z 2 (t) = 02, and hence g(t) = ciii(t) + C2Z2(t) < 0. When z\(t) = and 
z 2 (t) > 0, ii(t) = and y2(i) = 0, from which one has yi(i) = —Q\ and i2(i) = 
62 — a6\ < 0. Because in this case z\[t) = and Z2(t) < 0, g(t) < follows. 
When Z2 (t) = and z\(t) > 0, one can similarly argue that g(t) < 0. This 
shows that g(t) = for t > 0, as desired. 

We next demonstrate (D.3). Let Z be a two-dimensional SRBM with 
data (6,T,R) having a given initial point Z(0) = x E By Definition 6, 
Z, together with the associated pair (X, Y), satisfies (1.1)— (1.4). For each 
r > and each t > 0, set 

X r m = -X{rt), Y r (t) = -Y(rt), Z r (t) = -Z{rt). 

/v* ty* ty* 

By the functional strong law of large numbers (FSLLN) for Brownian mo- 
tion, almost surely, 

(D.4) lim sup \X r (s) - x(s)\ = for each t > 0, 

r ^°°0<s<t 

where x(s) = (9is,92s)' for s > 0. Fix a sample path that satisfies (D.4) and 
let {r n } C M+ be a sequence with r n — > 00. The FSLLN (D.4) implies that 
{X Tn } is relatively compact in C (M + , M) , the space of continuous functions 
on M + endowed with the topology of uniform convergence on compact sets. 
By the oscillation inequality, {(Y rn ,Z rn )} is also relatively compact. 

Let (y,z) be a limit point of {(Y rn ,Z rn )}. It is not difficult to show that 
(y,z) is a fluid path associated with the data (9,R) that satisfies z(0) = 0. 
It follows from the first part of the proof that z(t) =0 for t > 0. This is the 
unique limit point with z(0) = 0. Therefore, almost surely, 

lim -Z(r) = lim Z r (l) = 0, 

r— ¥00 t r—too 

which proves (D.3). □ 
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